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(57) Apparatus, methods, systems and computer 
program products are disclosed that determines what 
information presented on a computer display screen to 
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mation and the user's interest to select additional infor- 
mation for the user. This additional information better 
matching the user's interest. 
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Description 

Background of the Invention 

Fieid of the Invention 

This invention relates generally to the field of com- 
puter-human user interface technology and more par- 
ticularly to a method, apparatus, system and computer 
program product for allowing a computer to automati- 
cally determine what aspect of the computer's operation 
has the user's Interest and to optimize that aspect. 

Background 

Human/Computer Interaction 

An important characteristic of modem computing 
systems is the interface between the human user and 
the computer. Early interactive interfaces were text 
based wherein a user communicated with the computer 
by typing a sequence of characters on a keyboard and 
the computer communicated with the user by displaying 
characters on an output device-commonly a display 
screen. These input characters specified a command to 
the computer's operating system or to an application 
program executing on the computer. This command in- 
voked program logic to perform a given operation. Mo- 
dem computer systems use a graphical user interface 
(GUI) to simplify the interaction between a user and a 
computer. A GUI equipped computer communicates 
with a user by displaying graphics, including text and 
icons, on a display screen and the user communicates 
with the machine both by typing in textual information in 
response to dialogs and by manipulating the displayed 
icons with a pointing device, such as a mouse. 

Many modem GUIS provide a window environment. 
In a typical window environment the graphical display 
portrayed on the display screen is arranged to resemble 
the surface of an electronic "desktop" and each appli- 
cation program running on the computer is represented 
as one or more electronic "paper sheets" displayed as 
rectangular regions on the display screen. These rec- 
tangular regions are called "windows". Each window 
may include a multitude of panes. Each pane being an 
area for a particular type of information (textual, still im- 
age, moving image, etc.). 

Each window displays information generated by an 
associated application or system program. Further, 
there may be several windows simultaneously present 
on the desktop with each containing information gener- 
ated by a program. A program presents information to 
the user through each window by drawing or "painting" 
images, graphics or text within the window. The user can 
also move a window to a different location on the display 
screen and change its size and appearance to arrange 
the desktop in a convenient manner. The user commu- 
nicates with the program by "pointing at" objects dis- 
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played in the window with a cursor controlled by a point- 
ing device and manipulating the objects as desired. In 
some cases the program requests additional informa- 
tion from the user in response to a manipulation. This 

s request is presented as a "dialog" that allows the user 
to provide the requested information to the dialog from 
the keyboard. 

Each window typically includes a number of stand- 
ard graphical objects such as sizing boxes, buttons and 

10 scroll bars. These features represent user interface con- 
trols that the user can manipulate with the pointing de- 
vice. When the controls are selected or manipulated, the 
GUI invokes program logic in the underlying program to 
effect a corresponding command. 

IS One characteristic of a GUI is that the GUI is only 
responsive to a user's explicit manipulation of the point- 
ing device or keyboard. In the case of a mouse, the user 
physically moves the mouse device and a cursor on the 
display moves accordingly. Some pointing devices ac- 

20 tually track the user's gaze and move the cursor to 
where the user "looks" on the display screen. However, 
even with the gaze tracking (eye tracking) devices, the 
GUI only responds to the user's explicit commands 
whether that command be a button press, a blink, or a 

25 shift of view The computer remains a tool that the user 
operates by issuing explicit commands. 

In contrast, humans have the ability to make infer- 
ences by looking at another human's eyes. Pupils dilate 
when people see something attractive. People look at 

30 what they are interested in and stare at things they find 
interesting. Also, human eye movements reflect thought 
processes. Thus, humans observe what other persons 
do with their eyes and make inferences as to what that 
other person is interested in and/or thinking. 

3S The prior art in computer-human interfaces does 
not determine the user's immediate interest. Prior art 
computer-human interfaces simply respond to a user's 
command, whether input by typing the command at a 
keyboard, by manipulating a mouse to move a cursor, 

40 or by using a gaze tracking device to move a cursor. 
Thus, the computer is unable to detect or anticipate what 
characteristic of the computer's operation is of most in- 
terest to the user at any given time. 



Most gaze tracking devices operate based upon the 
principal that the direction of a person's gaze is directly 
related to the relative positions of the pupil and the re- 
flection of an object off the cornea (gaze tracking is often 
termed eye tracking). These devices often include im- 
age processing capabilities that operate on a video im- 
age of an eye to determine the gaze direction of the eye. 
These image processing capabilities are enhanced by 
using the bright eye affect. 

The bright eye affect is a result of the highly reflec- 
tive nature of the retina. This characteristic of the retina 
means that a significant amount of the light that enters 
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an eye Is reflected back through the pupil. Thus, when 
light shines into an eye along the axis of a camera lens, 
the retina reflects a significant portion of the light back 
to the cannera. Hence, the pupil appears as a bright disk 
to the camera. This affect allows the pupil to be more 
readily imaged from a video of an eye. 

Other methods exist for gaze tracking. Some incor- 
porate having two video cameras, one for tracking head 
movement and the other for measuring a reflection off 
of the eyes. Other mechanisms involve measuring elec- 
tric potential differences between locations on different 
sides of an eye. High accuracy devices are very intru- 
sive on the user and require that the user's head be held 
in a fixed position or that the user wear special equip- 
ment to track the eye. 

Recently, an eyegaze eyetracking system has been 
developed as described in The Eye gaze Eyetracking 
System - Unique Example of a Multiple, Use Technolo- 
gy, 4th Annual 1994 IEEE Dual-Use Technologies and 
Applications Conference, May, 1 994. This system com- 
prises a video camera located below a computer display 
that monitors one of the user's eyes. The device also 
contains an infrared light emitting diode (LED) located 
at the center of the camera's lens to maximize the bright- 
eye affect. Image processing software on the computer 
computes the user's gazepoint on the display sixty times 
a second with an accuracy of about a quarter inch. 

Gaze tracking devices have been used for weapon 
control, operator training, usability analysis, market re- 
search, and as an enablement for the disabled. Howev- 
er, gaze tracking devices have not been used to deter- 
mine what characteristic of a computer's operation in- 
terests the computer user at a particular time or to allow 
the computer to adapt to a user's interest as demonstrat- 
ed by where on the display screen the user is looking. 

Text to Speech 

Many modern computers now provide text-to- 
speech capability. This capability processes text strings 
and produces understandable audio speech from the 
computer's audio output device (headphones or speak- 
er). This capability allows a computer to present an au- 
dio version of a text string to a computer user. 

Problems with Downloading Information 

The background of the World Wide Web (WWW) 
and WWW browsers are well described by reference to 
the first chapter of Instant HTML Web Pages, by Wayne 
Ause. Ziff-Davis Press, ISBN 1-56276-363-6, Copyright 
1995, pages 1 -15, hereby incorporated by reference as 
illustrative of the prior art. 

Using the Internet, a computer user has access to 
an immense amount of information. However retrieving 
this information over the Internet often takes significant 
time because of the limited bandwidth of the communi- 
cation channel. The bandwidth is limited by many fac- 
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tors. Some of these factors are the bandwidth of the 
communication link from the user's computer to the In- 
ternet, the bandwidth of the communication link from the 
information provider's computer to the Internet, the ex- 
s istence of other communication traffic on these links, 
and the bandwidth of the Internet itself. Often, the pri- 
mary bandwidth limitation is at the user's computer. 

This bandwidth limitation at the user's computer is 
exacerbated because multiple data streams often flow 

10 across this limited communication link. If the user is in- 
terested in a particular data transfer, these additional da- 
ta streams utilize bandwidth that would otherwise be 
available to the data stream-of-interest to the user. This 
results in a decreased data transfer rate of the data 
stream-of-interest. 

Prior art WWW browsers, for example, generally at- 
tempt to equally allocate bandwidth to all the data trans- 
fers directed towards visible views in a window. Al- 
though this approach is clearly better then simply se- 

20 quentially retrieving data for each view, this approach 
delays retrieving data that is of the most interest to the 
user because the available channel bandwidth is divid- 
ed between the data streams supplying data to the 
views. Thus, the user must wait an additional time be- 

25 cause of uninteresting information using bandwidth that 
could have been applied to the information of interest. 

During the transmission of large amounts of data, a 
program generally provides some indication of the 
progress of the transmission. This indication is provided 

00 by indicators such as bar indicators, numerical percent- 
age indicators, or in the case of images often just the 
amount of detail available in the displayed image. While 
waiting for the transfer to complete, the user often 
watches the progress of the indicator or of the partially- 

35 filled Image. 

As mentioned above, one problem with the prior art 
is that a user has little control over the bandwidth allo- 
cated to the data stream used to download information. 
Further, even if an application should provide this control 

40 to the user, the user still must explicitly command the 
application to set the allocated bandwidth. 
The invention addresses these problems. 

Problems with Additional Data Associated with 
45 Images 

In print and computer hypertext documents, images 
such as pictures and illustrations, are often provided 
with additional information, such as captions explaining 

50 or enhancing the image. Those who view the image can- 
not look at the image and read an associated caption at 
the same time. Thus, the viewer's attention is diverted 
from the image while searching for, and reading, the as- 
sociated caption. Contrast this situation with a directive 

55 time-dependent medium, such as film or video, where 
a viewer is simultaneously presented with both visual 
and audio information. Audio captioning presents addi- 
tional information though an audio speaker allowing the 
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user to receive additional information auditorally without 
distracting the viewer's gaze from the Image of interest. 
Systems that allow a user to select which image to view, 
from a plurality of images, require the user to explicitly 
trigger the vocal caption. Thus, the user is again dis- 
tracted from looking at the image by the need to seek 
out and activate the caption. 

The invention addresses these problems. 

Problems with Small Text Displayed to a User 

People often have difficulty reading text on a com- 
puter display screen. Often this is due to vision difficul- 
ties. Thus, the type used in WYSIWYG (what you see 
is what you get) applications Is often too small for com- 
fortable reading at the display distance. Further, pub- 
lishers use different type sizes as a layout tool that indi- 
cates importance. Thus, there is a large variation in text 
size and screen space used between the largest head- 
line text and the text of an article. To address this prob- 
lem, some applications allow the WYSIWYG text to be 
magnified. Examples of word processing programs that 
provide this capability are Microsoft's Word® and Ado- 
be's FrameMaker(g) programs. However, these pro- 
grams require the user to explicitly specify, either direct- 
ly or indirectly, the desired magnification factor Further, 
the magnification process reduces the amount of the 
page that can be displayed on the computer display at 
the same time because the percentage of the page that 
is displayed to the user is reduced when the page is 
magnified. This problem is exacerbated with applica- 
tions that display WYSIWYG versions of newspapers 
and magazines because these applications generally 
attempt to maintain the WYSIWYG page layout and the 
displayed page is uniformly magnified. To see the entire 
page, the article text is generally reduced to unreadable 
size. 

The page layout of newspapers and magazines is 
important. To attract the interest of a large number of 
readers, the publishers of newspapers present a large 
number of articles on the first few pages. One way to 
increase the number of articles on a page is to decrease 
the amount of space used for the article. In a traditional 
newspaper, this is accomplished by moving subsequent 
parts of the article to different pages. This allows the 
reader to quickly scan articles that the editor believes to 
be most relevant and to read in depth those articles that 
the reader finds interesting. Further, where articles are 
placed on a page Influences the order that articles are 
viewed. Electronic newspapers have these same char- 
acteristics. 

Additionally, electronic newspapers, like traditional 
newspapers, use different type styles and sizes to indi- 
cate the relative importance of headlines and subhead- 
ers. Thus, there is a wide disparity between the largest 
and smallest text displayed to the reader Moreover, 
even large computer displays have a smaller display ar- 
ea than is available to a traditional newspaper thus re- 
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ducing the area available to the publisher for articles. 

Nevertheless, a computer display must often carry 
the same amount of information as a newspaper Thus, 
mapping the content of a newspaper onto a display 
5 screen reduces the size of the type used for the articles 
to the point where the text of the article is extremely dif- 
ficult to read. Further, the magnification method used by 
word processing programs for globally expanding the 
displayed text does not work well when presenting many 
10 articles on a page because magnifying the entire page, 
and providing a limited view into the page distances the 
structure of the page from the viewer Thus, globally ex- 
panding the text is incompatible with presenting as 
many articles as is desired on the page. Further, globally 
IS expanding the page also expands the larger title and 
headline text more than is needed to make this text read- 
able and at a cost of consuming undue display space 
that could otherwise be used to present additional infor- 
mation. Thus, there is a need for a mechanism that op- 
20 timizes the text size for a reader while still preserving 
the structural indications provided by the page layout. 
The invention addresses these problems. 

Problems with Selecting Relevant information for a 
25 User 

Another aspect of electronic newspapers, briefly 
mentioned above, is that of selecting information con- 
tent for the newspaper Information content includes 

30 both articles about particular items of interest and ad- 
vertising information. Information content is a major rea- 
son why people select different paper-based magazines 
and newspapers. These traditional information provid- 
ers present the information that they believe interest 

35 their readers. Traditional newspapers and magazines 
are static once printed. Thus, each edition is the same 
for all those who read it and each copy of a particular 
edition distributed to a particular region has the same 
articles, the same layout and the same advertising as 

"^0 all the other copies distributed to the region. This adver- 
tising and information content can be customized to the 
particular region. However, this regionalization can only 
be carried so far as it is extremely expensive to custom- 
ize a newspaper or magazine to the particular individual 

45 interests of each reader Thus, some of the information 
selected for a region will not interest some readers. 

Intangible electronic newspapers need not be con- 
strained by the above mentioned limitations inherent in 
using a tangible paper medium. However, electronic 

so newspapers still target most advertising and information 
content to a particular market and not to the particular 
interests of an individual reader Even where the reader 
of an electronic publication is provided with a means to 
customize the content of the electronic paper the user 

55 must explicitly specify the content. Further, by explicitly 
specifying the content, the user may not be presented 
with other related information that falls just outside of 
the specification but that could be of interest to the read- 
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The present invention provides an economical, ap- 
paratus, method, system and computer program prod- 
uct for providing enhanced facilities to computer users. 
This invention provides a way for a computer to monitor 
the user to determine what aspect of the computer op- io 
e ration the user is interested in and to respond accord- 
ingly. 

One aspect of the invention is a computer controlled 
method for presenting information to a user of a compu- 
ter. The method makes use of a display device and a is 
gaze-tracking device. The gaze-tracking device deter- 
mines the user's gaze position on the display device. 
The first step of the method displays a plurality of cate- 
gorized Information on the display device. It then moni- 
tors the gaze position to determine the user's level of 20 
interest in the plurality of categorized information dis- 
played on the display device. The Invention also re- 
trieves one or more topics that classify the plurality of 
categorized Information. Using these topics and the lev- 
el of interest the Invention determines a correlation that 2S 
is used to select a new plurality of categorized informa- 
tion. 

Another aspect of the invention discloses an infor- 
mation presentation apparatus that presents informa- 
tion on a display device to a user. This apparatus In- 30 
eludes a central processor unit, a memory and a gaze- 
tracking device. The gaze-tracking device determines a 
gaze position on the display device. The apparatus also 
includes a display mechanism that displays a plurality 
of categorized information on the display device. Addi- 35 
tionally the apparatus uses a monitoring mechanism to 
monitor the gaze position to determine the user's level 
of interest in the plurality of categorized information pre- 
sented on the display device. The apparatus also In- 
cludes a retrieval mechanism that retrieves topics that 40 
classify the categorized information displayed on the 
display device. These topics and the level of Interest of 
the user are provided to a correlation mechanism that 
determines a correlation between the topics and level 
of interest. Lastly, the apparatus includes a selection 4S 
mechanism that selects a new plurality of categorized 
information based on the correlation. 

Yet another aspect of the Invention is an information 
presentation system that presents information on a dis- 
play device to a user. This system Includes a gaze-track- so 
ing device. The gaze-tracking device determines a gaze 
position on the display device. The system also includes 
a display mechanism that displays a plurality of catego- 
rized information on the display device. Additionally the 
system uses a monitoring mechanism to monitor the ss 
gaze position to determine the user's level of interest in 
the plurality of categorized information presented on the 
display device. The system also Includes a retrieval 



mechanism that retrieves topics that classify the cate- 
gorized information displayed on the display device. 
These topics and the level of interest of said user are 
provided to a correlation mechanism that detennines a 
correlation between the topics and level of interest. Last- 
ly, the system includes a selection mechanism that se- 
lects a new plurality of categorized information based 
on the correlation. 

A final aspect of the invention discloses a computer 
program product having computer readable code em- 
bodied in a computer usable storage medium. When ex- 
ecuted on a computer, the computer readable code 
causes a computer to effect a display mechanism to dis- 
play a plurality of categorized Information on a display 
device. Further the invention contains code that effects 
a monitoring mechanism, a retrieval mechanism, a cor- 
relation mechanism and a selection mechanism having 
the same functions as the system described above. 

The foregoing and many other objects and advan- 
tages of the present Invention will no doubt become ob- 
vious to those of ordinary skill in the art after having read 
the following detailed description of the preferred em- 
bodiments which are illustrated in the various drawing 
figures. 

Description of the Drawings 

Figure 1 illustrates a portion of a computer sys- 

tem, including a CPU and a conven- 
tional memory in which the present in- 
vention may be embodied; 

Figure 2 Illustrates a display device fitted with 

gaze tracking equipment; 

Figure 3 illustrates aspects of a gaze position 

in accordance with a preferred em- 
bodiment; 

Figure 4 illustrates the process for determining 

a gaze position as used in accordance 
with a preferred embodiment; 

Figure 5 illustrates the operation of the Inven- 

tion to allocate bandwidth to an area 
of interest in accordance with a pre- 
ferred embodiment; 

Figure 6 Illustrates the process used to change 

the bandwidth of a data stream based 
upon a gaze position in accordance 
with a preferred embodiment; 

Figures 7a&b illustrate audio captioning in accord- 
ance with a preferred embodiment; 

Figure 8 illustrates extraction of an image cap- 

tion from a page of text in accordance 
with a preferred embodiment; 

Figures 9 illustrates the process used to Imple- 

ment captioning in accordance with a 
preferred embodiment; 
. FIgur 10 illustrates the form of an electronic 

newspaper; 

Figure 1 1 illustrates text magnification and page 
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layout In accordance with a preferred 
embodiment; 

Figure 12 illustrates text magnification and page 

layout in accordance with a second 
preferred embodiment; s 

Figur 13 illustrates the process of expanding 

text in response to the users interest 
in the text in accordance with a pre- 
ferred embodiment; 

Figure 14 illustrates the process for adjusting io 

the layout of a display as a result of 
expanded text In accordance with a 
preferred embodiment; 

Figure 15 Illustrates a possible first page of an 

electronic newspaper showing arti- '5 
cles and an advertisement in accord- 
ance with a preferred embodiment; 

Figure 16 illustrates a possible second page of 

an electronic newspaper showing in- 
formation determined to be of Interest 20 
to the reader In accordance with a pre- 
ferred embodiment; and 

Figure 17 illustrates the process used to evalu- 

ate the information of interest to a 
reader and to select new information 2S 
matching the readers interest in ac- 
cordance with a preferred embodi- 
ment. 

Description of the Preferred Embodiments 30 
Notations and Nomenclature 

The following "notations and nomenclature" are 
provided to assist in the understanding of the present 35 
invention and the preferred embodiments thereof. 

Advertisement - Information provided about a 
commercial product or service with the purpose of in- 
forming the viewer about the product or service so as to 
lead to a commercial transaction. A type of article. 40 

Article - A complete piece of writing often identified 
with a title. 

Bandwidth - The amount of information that can be 
passed across a communication channel in a given pe- 
riod of time (usually designated in reference to a sec- 45 
ond). 

Dialog - A specialized window that is used to obtain 
additional information from the user. A dialog is often 
used to obtain options and parameters that are compu- 
ter dependent. A good example is a print dialog that is 50 
evoked by a print menu command. The print dialog al- 
lows the user to specify what printer options are to be 
used for a particular print job. Generally the dialog al- 
lows the user to specify specific parameters and then to 
either affirm or cancel the command that evoked the di- ss 
alog. If the user cancels the command, the dialog win- 
dow is removed and the command that evoked the dia- 
log is aborted. If the user confirms the command the us- 



er provided information acquired by the dialog is used 
in the execution of the command that evoked the dialog. 

E-mail system - Electronic mail system. A system 
of computers generally connected by a network that al- 
low a sender (being a user of a first computer) to com- 
pose and send data making up a message to a recipient 
(being a user of either the first computer or of a second 
computer). 

Graphical User Interface (GUI) - A user interface 
that allows a user to interact with a computer display by 
pointing at selectable control areas on the display and 
activating a command or computer operation associat- 
ed with the selectable control area. GUIs are well known 
in the art. 

Gaze position - An area of interest on the screen 
providing a boundary of the user's gaze over a limited 
period of time. 

Gaze coordinates - The coordinates that represent 
the intersection of the user's gaze with the display 
screen over a limited period of time. 

Gaze coordinates (raw) - The coordinates that 
represent the instantaneous intersection of the user's 
gaze with the display screen. 

Image - Any information displayed on a display 
screen such as, but not limited to, pictures, drawings, 
illustrations, text, and video. An image generally dis- 
played in a view contained in a window A still image is 
a picture. A moving image Is comprised of a number of 
frames of still images that are played in sequence similar 
to a video 

Pointing device - A device responsive to a compu- 
ter user's input that moves an indicator on a computer 
display screen. Such an indicator has an active point 
such that if the pointing device is activated (e.g., by a 
button push for a mouse device) a command associated 
with the selectable control area covered by the active 
point is invoked. Pointing devices are generally used 
with graphical user interfaces. 

Selectable control area - An area on a computer 
display that is sensitive to activation of a pointing device. 
On activation of the pointing device over the selectable 
control area, a command or computer operation asso- 
ciated with the selectable control area is invoked. Most 
computer systems that provide a Graphical User Inter- 
face (GUI) also provide other methods for invoking 
these commands or computer operations such as key- 
board function keys or command lines. 

URL - A Uniform Resource Locator. URLs are used 
to access information on the World Wide Web. 

View - An area in a window where information is 
provided. 

Window - An area, usually rectangular, on a com- 
puter display screen controlled by an application. 

Procedure - A self-consistent sequence of steps 
leading to a desired result. These steps are those re- 
quiring physical manipulation of physical quantities. 
Usually these quantities take the form of electrical or 
magnetic signals capable of being stored, transferred, 
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combined, compared, and otherwise manipulated. 
These signals are referred to as bits, values, elements, 
symbols, characters, terms, numbers, or the like. It will 
be understood by those skilled in the art that all of these 
and similar terms are associated with the appropriate 
physical quantities and are merely convenient labels ap- 
plied to these quantities. 

Overview 

The manipulations performed by a computer in ex- 
ecuting opcodes are often referred to in terms, such as 
adding or comparing, that are commonly associated 
with mental operations performed by a human operator. 
In the present invention no such capability of a human 
operator is necessary in any of the operations described 
herein. The operations are machine operations. Useful 
machines for performing the operations of the invention 
include programmed general purpose digital computers 
or similar devices. In all cases the method of computa- 
tion is distinguished from the method of operation in op- 
erating a computer. The present invention relates to 
method steps for operating a computer in processing 
electrical or other (e.g., mechanical, chemical) physical 
signals to generate other desired physical signals. 

The invention also relates to apparatus for perform- 
ing these operations. This apparatus may be specially 
constructed for the required purposes or it may com- 
prise a general purpose computer as selectively activat- 
ed or reconfigured by a computer program stored in the 
memory of a computer. The procedures presented here- 
in are not inherently related to a particular computer or 
other apparatus. In particular, various general purpose 
machines may be used with programs written in accord- 
ance with the teachings herein, or it may prove more 
convenient to construct more specialized apparatus to 
perform the required method steps. The required struc- 
ture for a variety of these machines will appear from the 
following description. Also, the invention may be em- 
bodied in a computer readable storage medium encod- 
ed with a program that causes a computer to perform 
the programmed logic. 

Operating Environment 

Figure 1 illustrates a computer system referenced 
to by the general reference character 1 02, configured to 
support the invention. The system 102 includes a proc- 
essor 133 having an Input/Output ("I/O") section 135, a 
central processing unit ("CPU") 1 37 and a memory sec- 
tion 1 39. The I/O section 1 35 is connected to a keyboard 
141, a disk storage unit 143, a network interface 145 to 
provide access to a network 117, a display unit 147, a 
pointing device 148, a gaze-tracker device 155, a 
speaker 157 and a CD-ROM drive unit 149. The CD- 
ROM unit 149 can read a CD-ROM medium 151 that 
typically contains a plurality of programs 153 and data. 
The CD-ROM 149 and the disk storage unit 143 com- 
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prising a filestorage mechanism. One skilled in the art 
will understand that the filestorage mechanism may 
comprise read only memory, RAM or other storage tech- 
nology that allows a computer to access data. Such a 
5 computer system is capable of executing programmed 
logic that embodies the invention. 

Gaze Traclcer Operation 

Figure 2 illustrates a gaze tracking device 201 at- 
tached to a computer display device 203. As illustrated, 
the gaze tracking device 201 Is mounted below the com- 
puter display 203 and comprises a video lens 205, with 
an infrared LED 207 mounted on the center of the lens 
205. One skilled in the art will understand that one of 
many different types of gaze tracking devices can be 
used with the invention. The computer display device 
203 has a display area 209 that the computer can ac- 
cess to display information. The computer generally cre- 
ates a plurality of windows 211 and 213 for this informa- 
tional display. As the user looks at the display area 209, 
the gaze tracking device determines the users gaze po- 
sition. The gaze coordinate (raw) is an ordered pair of 
values providing the immediate two dimensional coor- 
dinates of where the user's gaze is positioned on the 
screen. A number of gaze coordinates can be used to 
develop a gaze position that is a two dimensional coor- 
dinate of a time weighted average of the gaze coordi- 
nates. The invention need not have precise gaze coor- 
dinates. In fact the gaze coordinates need only be ac- 
curate to approximately a 1/4 inch. One skilled in the art 
will understand that the gaze position could also be re- 
turned as a rectangle, polygon, circle or other shape rep- 
resenting an area of interest. Further one skilled in the 
art will understand that the gaze tracking device^201 
may be configured to provide the gaze position'only 
when the user has shown interest in a displayed area 
for a sufficient time, or that application or system pro- 
gram logic may be utilized to detect when the user has 
shown an interest by monitoring the gaze coordinates 
over some period of time. The gaze position is devel- 
oped when the system detects that the user has become 
interested In a particular area of the display. The system 
detects this condition by recognizing that the user's 
gaze has been limited to a particular region of the dis- 
playable area 209 for a period of time. 

Figure 3 illustrates aspects of a gaze position de- 
termined from gaze coordinates returned from a gaze 
tracking device. The dashed rectangle labeled as 301 
indicates the boundaries of an image (not shown). Be- 
cause the gaze tracking device has an uncertainty and 
because the user's gaze constantly moves (because of 
involuntary eye movements) even when focused on an 
area of interest, this embodiment of the invention pro- 
vides the gaze position as an area 305 where the user's 
gaze is fixed. The provided area 305 indicates that the 
user's gaze did not leave the specified area for a given 
time such as a half second. The application program that 
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uses the returned circular area 305 is able to determine 
that the user's gaze is well within the innage 301 because 
the provided circle is connpletely contained within the ar- 
ea of the image. If the user has interest in the entire im- 
age, the gaze would be more diffused resulting in a larg- 
er circle. For example, in comparing the image labeled 
301 with an image 311, the user viewing the image 301 
has focused in on a particular aspect of the image as 
indicated by the circle labeled as 305. The user viewing 
the image 311 has indicated more interest in the totality 
of the image as compared to a particular aspect of the 
image as indicated by the circle labeled as 309. One 
skilled in the art will understand that a rectangle or other 
area indication can be used as well as a circle. 

In another embodiment of the invention, the gaze 
tracker returns the gaze position as a point. For exam- 
ple, a dashed rectangle labeled as 303 indicates the 
boundaries of a different image (again this image Is not 
shown). In this embodiment, the gaze tracker, after 
processing gaze movements over some period of time, 
simply returns a point that is a time weighted average 
of the position of the gaze. This point is indicated by the 
arrow labeled as 307. 

Figure 4 illustrates the process used to develop a 
gaze position from a plurality of gaze coordinates. The 
process starts at a terminal 401 . Next at a step 403, the 
process performs any necessary setup or initialization 
steps. This Includes initializing timer program logic to 
periodically gather instantaneous gaze coordinate data 
from the gaze tracker apparatus, acquiring user prefer- 
ence data and other one-time initializations. Next at a 
step 405, the process receives a plurality of gaze coor- 
dinates (raw). These gaze coordinates (raw) Indicate 
the position of the user's gaze over a given time. After 
a sufficient number of gaze coordinates (raw) are gath- 
ered, the process examines the gaze coordinates (raw) 
for a focus pattern at a branch point 407. A focus pattern 
occurs when the gaze coordinates (raw) are all within 
some boundary without gaze coordinates (raw) being 
statistically far from the others. If the gaze coordinates 
(raw) do not show a focused gaze, the gathering of in- 
stantaneous gaze coordinates (raw) continues as 
shown by an arrow 406. If at the branch point 407 a focus 
pattern is found then a step 409 determines an area of 
focus that contains the statistically significant gaze co- 
ordinates (raw). Finally In a step 411 the gaze position 
is made available to the program logic for use. Finally, 
the process repeats as shown by an arrow 412. In the 
case where the gaze position is presented as an area 
Instead of a simple coordinate pair, the receiving pro- 
gram logic determines which view is overlapped by the 
gaze position. 

There are various well known methods in the art to 
present the gaze position to program logic such as an 
application program. These include providing an excep- 
tion to an application, sending an interprogram message 
containing the gaze position to the application and many 
other methods. 



An executing computer program can modify its op- 
eration to best suit the user's Interest by using a gaze 
tracking device to determine which area of a display 
screen is the object of a user's attention. Specific em- 
5 bodiments using this capability of the gaze tracker are 
described below. 

Gaze Tracker Mediated Downloading 

10 As mentioned above, prior art communication pro- 
grams such as WWW browsers and other data transfer 
programs that provide multiple data streams do not al- 
locate bandwidth depending on the user's specific inter- 
est. At best, these programs will allocate bandwidth to 
IS images and data that Is presented to the user on the 
display device. The invention enables a computer to de- 
tect which data stream the user is interested in and to 
allocate more bandwidth to that data stream without an 
explicit command from the user. 

Figure 5 illustrates how a preferred embodiment of 
the invention is used in a WWW browser. The browser 
application displays a window 501 on the display device. 
The user Invokes a URL to present a webpage contain- 
ing Information encoded in HTML in the window 501. In 
this particular example, the webpage shows a plurality 
of three areas of text 503, 511 and 517 along with a plu- 
rality of areas of images 505, 513 and 519. Each of these 
images are displayed in a view. Further, these images 
are constructed from "large data entities". Each large 
date entity is composed of a large amount of data that 
is used to define the entity. Examples of large data en- 
titles, among others, are high resolution graphical imag- 
es, sound and video entities. A plurality of shaded areas 
507, 509 and 521 of the images 505, 513 and 519 indi- 
cate the amount of the image that has been downloaded 
at some particular point In time prior to completion. If the 
user is Interested In the Image labeled as 513, the user 
will watch that image 513 fill. Thus, a gaze position area 
515 intersects the image 513 thus identifying the image 
513 as an area of interest. A complementary approach 
to determining which image 505, 51 3 or 51 9 is of interest 
is to have the text 503, 511 or 517 associated with each 
image. Thus, when the user reads the text 511 linked 
(associated) with an image 513 a gaze position 512 is 
found on the text. Once the user shows Interest in the 
text, the application can properly assume that the area 
of interest is the linked image 513. 

Once the area of interest is determined, the system 
obtains the bandwidth allocated to the area of Interest 
and, if possible, increases the bandwidth allocation of 
the data stream directed to the area of interest using 
techniques well understood In the art. 

Figure 6 illustrates the process used to allocate 
bandwidth to a data stream supplying data to an area of 
interest. The process starts at a terminal 601. At a step 
603 the process receives the gaze position from a gaze 
tracking device as described above. Then at a step 605 
the gaze position is used to determine what Information 
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or view on the display device intersects with the gaze 
position. Next in a decision block step 607, the process 
checks whether the displayed information that inter- 
sects the gaze position is relevant. Information that is 
relevant comprises information that is In the process of 
being transferred to the display device. In other words, 
a completely loaded image is not relevant and has no 
bandwidth allocation because there is no need to in- 
crease the bandwidth for an image that has been com- 
pletely transmitted. Contrast this situation where there 
is zero bandwidth allocated to a yet-to-be- loaded image. 
Here the yet-to-be-loaded image has a bandwidth allo- 
cation, but the current allocation is zero whereas there 
is no bandwidth allocation at all for an image that has 
been completely downloaded. Text is another example 
of displayed information that usually is not relevant be- 
cause the user cannot read the text as fast as it is trans- 
mitted even with limited bandwidth. 

If in the decision block step 607 the displayed infor- 
mation is relevant then, in a step 609 the process deter- 
mines the area of Interest on the display and In a step 
611 increases the bandwidth allocated to the data flow 
associated with the area of Interest. Finally, the process 
completes through a terminal 613. 

However, if In the decision block step 607 the dis- 
played information is not relevant, the source of the dis- 
played information is checked in a decision block step 
615 to determine whether the source indicates that the 
displayed Information is associated with an area of in- 
terest. An example of this situation is where text is as- 
sociated with an image. If at the decision block step 615 
the source is not associated with an area of interest 
processing completes through the terminal labeled as 
613. However, If 615 the source is associated with an 
area of interest, the process 611 Increases the band- 
width of data flowing to that area of interest. Finally, the 
process completes through the terminal 613. 

In a preferred embodiment of a WWW browser, the 
process illustrated In Figure 6 determines what area of 
the display device the user is Interested in. If that area 
is a partially downloaded image, the invention increases 
the bandwidth allocated to the data flow for completing 
the image. If the area is a textual area whose defining 
source HTML provides an association with an image (for 
example, by providing an attribute within the paragraph 
element that associates a large data entity with the par- 
agraph) the invention increases the bandwidth allocated 
to the data flow for completing that specific image. 

Finally, embodiments utilizing the invention are not 
limited to WWW browsers. Any application that presents 
information that Is downloaded across a limited band- 
width network onto a display device can use the inven- 
tion. This includes detecting interest on data transfer 
status displays generated by programs such as, among 
others, FTP, Kermit, or x- y- and zmodem applications. 



Captioning 

Another preferred embodiment of the invention re- 
lates to captioning of images. Audio captioning detects 
5 the user's Interest in a displayed image and automati- 
cally provides additional audio information to the user 
about the displayed image while video captioning pro- 
vides additional video Information. 

Figure 7a illustrates captioning. An image of a 
10 phone 701 is displayed to the user If the user is inter- 
ested in the phone, the user looks at the Image 701 thus 
generating a gaze position from the gaze tracking de- 
vice. The invention then presents additional Information 
about the phone to the user. In some circumstances, the 
15 additional information is electronic information attached 
to the image by a data structure within the underlying 
application such as the one shown in Figure 7b. Such 
additional information may include text, text that Is con- 
verted to audio, recorded audio data, video data, or an- 
other image. 

In other circumstances the additional Information Is 
a textual caption 703 displayed with the image 701 but 
without a direct electronic linkage to the captioned im- 
age 701 . When the additional information is a displayed 
textual caption 703, the Invention detects that a textual 
caption 703 exists for the image 701, extracts that tex- 
tual caption 703 and passes the extracted text string to 
a text-to-speech process that presents the caption text 
to the user in audio form. Some page markup languages 
support a <CAPTION> tag. For example, both SGML 
and HTML support such a tag (note however, that al- 
though HTML supports a <CAPT10N> tag, It is currently 
only used for table captions and not for figures). Thus 
one of the preferred embodiments of the invention uses 
the <CAPTION> tag to detect the textual caption 703 of 
an image 701. 

All of these methods provide additional information 
to the user about an Image-of- interest. The audio meth- 
ods allow the user to maintain his/her attention on the 
Image while receiving additional information auditorally. 
Many of the other methods of presenting the additional 
Information result In the user removing her/his gaze from 
the image-of -interest. 

Figure 7b illustrates a representative data structure 
710 that can be used to associate additional electronic 
information with an image as mentioned above. An "Im- 
age ID" field 711 identifies the image displayed to the 
user. A "Pointer to Image Data" field 713 contains the 
image data, or data that directly or indirectly references 
the image data used to create the displayed image 701. 
A "Pointer to Additional Information" field 715 contains 
the additional electronic information data, or data that 
directly or indirectly references the additional electronic 
information data. This data structure is one of many that 
can be used by programmed applications that provide 
an explicit link between an image and additional infor- 
mation. Some programmed applications do not provide 
such an explicit linkage. 
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Figur 8 illustrates a window 801 displaying an ex- 
annple page 803. The page 803 includes text and an im- 
age 805 of a phone. The phone image 805 has an as- 
sociated textual caption 807. The image 805 and the 
caption 807 are offset from the text by a plurality of blank s 
areas 809, 811 and 813. These blank areas 809, 811, 
813 delimit the image 805 and its associated textual 
caption 807 from the rest of the displayed text. This sep- 
aration allows the underlying application program to de- 
termine a linkage between the image and its associated io 
textual caption. Further, even if the underlying pro- 
grammed application that presented the window 801 
does not have the facility to directly determine the link 
between the caption 807 and the image 805, other pro- 
grammed applications can scan the window 801. After is 
scanning the window these programmed applications 
then isolate and pass the caption text 807 through a text- 
to-speech facility to convert the caption text 807 to audio 
and present the audio to the user. 

Figure 9 illustrates the process used to present ad- 20 
ditional information . The process starts at a terminal 951 
when it receives a gaze position from a gaze-tracking 
device at a step 953. If at a decision block step 955 the 
gaze position does not intersect an image, the process 
completes through a terminal 957. If at the decision 2S 
block step 955 the gaze position intersects an image 
and if at a decision block step 959 the image has an 
electronic caption, the process accesses and presents 
the additional information at a step 961 . Finally, the proc- 
ess completes through the terminal 957. If at the deci- 30 
sion block step 959 the image does not have an elec- 
tronic caption but does have a displayed textual caption 
as determined in a decision block step 963, the process 
extracts the caption text and passes the resulting text 
string to a text-to-speech facility for auditory presenta- 55 
tion to the user in a step 965. Finally, the process com- 
pletes through the terminal 957. If in the decision block 
step 963 the image does not have a textual caption, or 
if such a caption cannot be determined, the process 
completes through the terminal 957. 40 

One skilled in the art will understand that auditory 
presentation of the additional information to the user is 
accomplished with techniques known in the prior art. 

Displayed Text Enlargement 45 

A page from an electronic newspaper is illustrated 
in Figure 10. This information is displayed on a compu- 
ter display screen generally in a window as discussed 
above. Here a window 1001 contains a headline 1003, so 
a plurality of four article titles 1 005, 1 009, 1 01 3 and 1 01 7 
and a partial text 1 007, 1 01 1 , 1 01 5, and 1 01 9 of the titled 
articles. A computer for displaying the electronic news- 
paper that is equipped with a gaze tracking device can 
rely on the normal reading pattern of a user That is, that ss 
the user first reads the major headline 1003, then the 
article title 1005, 1009, 1013 or 1017 followed by the 
article Itself 1007, 1011, 1015, or 1019. Thus, when the 



gaze tracker indicates that the user is reading an article 
title, the Invention starts expanding the article text by a 
magnification factor to an optimal size for the user. 

Figur 11 illustrates two possible embodiments of 
the invention. As in Figure 10 within a window 1101, a 
plurality of four article titles 1105, 1109, 1113, 1117 are 
displayed. The window 1101 also includes a headline 
1103. An article 1107 has been expanded by the inven- 
tion to a comfortable size for reading. This expansion 
has occurred without increasing the size of the window 
1101 within which the article 1107 is displayed. Howev- 
er, the view containing the article 1107 and the text of 
the article 1 1 07 have been expanded. To obtain the nec- 
essary window space to expand the view containing the 
article 1107 without interfering with the expanded view 
1107, the view containing an article 1115 along with its 
title 1113 have been reduced in size and shifted towards 
the edge of the window 1101. The articles associated 
with the titles 1109 and 1117 have been shifted com- 
pletely off of the window 1101 leaving only their associ- 
ated titles 1109 and 1117. 

It the computer detects that the user's gaze has 
moved from the expanded article 1 1 07, to a different title 
1113, 1109, or 1117, the text of the associated article is 
expanded. At the same time, the text and title of the pre- 
viously expanded article 1105, 1107 is slowly reduced 
so that the user's gaze is not attracted by the apparent 
movement caused by the reduction. In a preferred em- 
bodiment of the invention the user may utilize computer 
commands to cause the reduction and expansion of the 
articles and their associated title to be Instantaneous. 
Further, the expansion can be to a predetermined user 
preference. Finally, the expansion of the article can be 
dynamically determined by monitoring the user's read- 
ing speed of the article (by determining how the gaze 
position changes overtime) so as to maximize the read- 
ing speed. 

Another embodiment of the invention retains the 
original size of the view, but magnifies the text within the 
view by a magnification factor. This embodiment does 
not require the reduction and repositioning of the other 
articles as does the preferred embodiment. However, as 
the text expands within the limited bounds of the view 
the amount of text that can be displayed in the unmag- 
nlfied view is reduced. Further, the maximum magnifi- 
cation of the text is limited in that the essential structure 
of the left to right reading pattern (for Latin based lan- 
guages) should be retained for comfortable reading. Too 
much magnification of the text within a limited space re- 
sults in a column of words that is difficult to read. 

Figure 1 2 illustrates another approach to managing 
the information displayed In a window 1201 . Here, a title 
1205 and text of an article AA 1207 Is expanded and 
simply overlays a plurality of other articles 1211, 1215, 
and 1 21 9 In the window 1 201 . Because the other articles 
and titles are obscured, the text is reduced when the 
user's gaze leaves the window 1201 or when the user 
invokes a command function either by use of a pointing 
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device, a command line or other command invocation 
method. Because the magnified view overlays the other 
views, a preferred embodiment places the magnified 
view within a boarder 1217 to separate the magnified 
view from the other views. 

Figure 1 3 illustrates the process used to determine 
when to expand an article. The following description ref- 
erences Figure 1 0 as an illustrative example. The proc- 
ess starts at a terminal 1 301 . At this time the user's gaze 
has not focused in on a window 1001. When the user's 
gaze enters the window 1001, the process, at a step 

1305, monitors which text is read by the user. So long 
as the gaze position indicates that the user is reading 
one of the plurality of titles 1005, 1009, 1013, and 1017 
as determined by a decision block step 1307 the moni- 
toring step 1305 continues as indicated by an arrow 

1306. If the decision block step 1307 detects that the 
user's gaze changed to a subtitle or the plurality of article 
text 1007, 1011, 1015, 1019, the process starts to ex- 
pand the view and text of the contained article at a step 
1309. The process continues to monitor the user's gaze 
position at a step 1311 to determine the user's reading 
speed. Next at a step 1315, the process adjusts the size 
of the article text to maximize the user's reading speed 
possibly continuing the expansion of the view started in 
the step 1311. This adjustment uses a magnification fac- 
tor having a value that is a function of the reading speed. 
This magnification factor controls the view and the text 
within the view. Finally at a decision block 1317, the 
process determines whether the user's gaze position 
continues to intersect the view containing the article 
(that is, to verify that the user continues to read the ar- 
ticle). If at the decision block 1317 the user continues to 
read the article, the monitoring process of the step 1 31 1 
continues as indicated by an arrow 1313. However if at 
the decision block 1317, the user has stopped reading 
the article, the process starts reducing the size of the 
abandoned article at a step 1319 and the entire process 
is repeated as indicated by an arrow 1 321 . The invention 
allows the user to reacquire the abandoned article sim- 
ply by bringing the user's gaze back to the text of the 
abandoned article. This stops the reduction of the size 
of the article text and restarts the expansion of the article 
text at the step 1309 and leading to the text adjustment 
step 1315. 

In the enlarging and reduction steps above, the text 
is slowly enlarged and reduced so as not to be a distrac- 
tion to the user. Thus the change in size does not attract 
the user's gaze from the article being read. However, 
the invention also provides the user with commands that 
accelerate these scaling processes. Further, the maxi- 
mum size of an expanded article can be specified as a 
user preference. 

Figure 14 illustrates the process used to size the 
views impacted by the magnification of the view contain- 
ing the article being read by the user - the active view. 
The process starts at a terminal 1401. At a step 1403 
the process first determines the new position and size 



of the active view in response to the change of scale. 
Based on this new position and size, at a step 1405 the 
process determines which other views displayed in the 
window 1001 will be affected by the change in size and 
5 position of the active view. Then, at a step 1407, each 
affected view is determined and adjusted in a step 1409 
to make room for the enlarged active view (where the 
active view is expanding), or to take up room released 
by a reduced active view (where the active view is con- 
10 tracting). This process is repeated as indicated by an 
arrow 1411 for each impacted view. Once the impacted 
views are adjusted the process continues, as indicated 
by an arrow 1415, to adjust the size and position of the 
active view in a step 1417. Finally, the process com- 
15 pietes through a terminal 1419. Note that a reduction in 
the size of the active view occurs to optimize the user's 
reading speed. This is not the same reduction that re- 
sults when the user has stopped reading a view and that 
view contracts because it is no longer the active view. 

This aspect of the invention has been described as- 
suming normal English reading patterns (that is, top to 
bottom, left to right). One skilled in the art will under- 
stand that other reading pattems are contemplated by 
the invention. 

Interest Based information Presentation 

Figure 15 illustrates a typical electronic newspaper 
display The invention also applies to data search en- 
gine displays. Here, a number of articles 1507, 1511, 
1515 and 1519 along with their associated titles 1505, 
1509, 1513 and 1517 are displayed in views within a 
window 1501. Generally a major headline 1503 is also 
displayed along with a selection of advertising material 
1521. Because the information provider does not know 
what subjects interest the user, the information provider 
presents a mixture of articles. Here the "New Chemical 
Reaction" article 1507, and the "Quarks" article 151 9 are 
both science and technology related. The other two ar- 
ticles 1515 and 1511 are not. Each article and adver- 
tisement contains information that can be categorized 
in multiple ways. This categorization includes at least 
one topic classifying the information. These topics are 
developed and maintained by the information provider. 
Using gaze tracking, the Information provider can deter- 
mine the user's interest in each displayed article 1505, 
1509, 1513 and 1517 and advertisement 1521. Then, 
by using the topics categorizing the presented informa- 
tion, the information provider can dynamically adjust the 
selection of subsequent information presented to this 
user. In the example above, suppose the user read the 
scientific based articles 1507 and 1519 but did not 
spend any time reading the other articles 1511 or 1515 
or the advertisement 1521. The information provider 
populates the next page of information presented to the 
user with articles and advertisements that have similar 
topics as the previously read information. 

Figure 16 illustrates a possible second page of in- 
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formation. Again, the infornnation is provided within 
views contained in a window 1601. Now a plurality of 
articles 1607, 1611, 1615 and 1619 are all scientific or 
technology based, but with different levels of difficulty 
extending from articles of interest to the lay reader to 
those that are directed toward the advanced elemental 
particle physicist. Further, both a Major Scientific Head- 
line 1603 and an advertisement 1621 can be selected 
to be of interest to the user. This allows the information 
provider to narrowly target advertising and articles to 
each user. Again the information provider can continue 
to refine and narrow the selection of information pre- 
sented to the user on subsequent pages depending on 
the interest shown in a plurality of article titles 1605, 
1609, 1613, 1617, the time spent with reading each ar- 
ticle 1607, 1611, 1615 and 1619; and the time spent 
looking at the advertisement 1621 of the current page. 

Figure 17 illustrates the process used to select in- 
formation for presentation to a user. The process starts 
at a terminal 1701 after an initial selection of information 
is displayed to the user. Using the gaze position devel- 
oped as described above, a step 1705 monitors the us- 
er's reading pattern. Further a step 1709, determines 
the amount of interest shown by the user in the dis- 
played information. This interest is determined by meas- 
uring the user's reading speed, determining whether the 
user only skimmed the information or read the informa- 
tion in depth, and by measuring the amount of time spent 
with each article and advertisement. Then in a step 
1711, the process retrieves the topics associated with 
each displayed information and in a step 1713 corre- 
lates these topics with the user's interest. Next in a step 
1715, the process selects additional information based 
on this correlation. This selection of information is dis- 
played in a step 1717 for the user. Finally, the process 
completes through a terminal 1719. In this manner, the 
user is presented with a customized set of information 
that reflects the user's interest. 

Although the present invention has been described 
in terms of the presently preferred embodiments, one 
skilled in the art will understand that various modifica- 
tions and alterations may be made without departing 
from the scope of the invention. Accordingly, the scope 
of the invention is not to be limited to the particular in- 
vention embodiments discussed herein, but should be 
defined only by the appended claims and equivalents 
thereof. 



Claims 

1. A computer controlled method for presenting infor- 
mation to a user of a computer; said computer hav- 
ing a display device and a gaze-tracking device; 
said gaze-tracking device determining a gaze posi- 
tion on said display device; said computer control- 
led method comprising the steps of: 



(a) displaying a plurality of categorized informa- 
tion on said display device; 

(b) monitoring said gaze position to determine 
5. a level of interest of said user in said plurality 

of categorized information displayed on said 
display device; 

(c) retrieving one or more topics classifying said 
'fo plurality of categorized information; 

(d) determining a correlation between said top- 
ics and said level of interest; and 

(e) selecting a new plurality of categorized in- 
formation based on said correlation. 

2. The computer controlled method of claim 1 further 
comprising the steps of: 

20 

maintaining a correlation history associated 
with said user; and 

refining said correlation using said correlation 
2S history. 

3. The computer controlled method of claim 1 wherein 
said plurality of categorized information includes an 
advertisement: step (d) further comprises the step 

30 of determining an advertising correlation; and step 
(e) further comprises the step of selecting a new ad- 
vertisement based on said advertising correlation. 

4. The computer controlled method of claim I further 
3S comprising the steps of: 

measuring said user's reading speed for at 
least one of said plurality of categorized infor- 
mation; and 

40 

measuring a time period when said gaze posi- 
tion is within a display area containing one of 
said plurality of categorized information; said 
time period and said reading speed comprising 
correlation parameters. 

5. The computer controlled method of claim 4 wherein 
said correlation includes one or more of said corre- 
lation parameters. 

so 

6. An information presentation apparatus configured 
to present information to a user; said apparatus 
having a central processor Unit, a memory, a display 
device, and a gaze-tracking device for determining 

55 a gaze position on said display device; said appa- 
ratus comprising: 

a display mechanism configured to display a 
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7. 



plurality ol categorized information on said dis- 
play device; 

a monitoring mechanism configured to monitor 

said gaze position to determine a level of inter- s 11. 

est of said user in at least one of said plurality 

of categorized information displayed on said 

display device; 

a retrieval mechanism configured to retrieve io 
one or more topics classifying said categorized 
information displayed on said display device; 

a correlation mechanism configured to deter- 
mine a correlation between said topics and said is 
level of interest of said user; and 

a selection mechanism configured to select a 
new plurality of categorized information based 
on said correlation. 20 

The information presentation apparatus of claim 6 
further comprising: 

a history mechanism configured to maintain a 
correlation history associated with said user; 
and 



wherein said correlation mechanism additionally 
uses one or more of said correlation parameters to 
determine said correlation. 

An information presentation system configured to 
present Information to a user; said system having a 
display device, and a gaze-tracking device for de- 
termining a gaze position on said display device; 
said system comprising: 

a display mechanism configured to display a 
plurality of categorized Information on said dis- 
play device; 

a monitoring mechanism configured to monitor 
said gaze position to determine a level of inter- 
est of said user in at least one of said plurality 
of categorized information displayed on said 
display device; 

a retrieval mechanism configured to retrieve 
one or more topics classifying said categorized 
information displayed on said display device; 

a correlation mechanism configured to deter- 
mine a correlation between said topics and said 
level of interest of said user; and 



a refinement mechanism configured to deter- 
mine a refined correlation using said correlation 
history. 



30 



a selection mechanism configured to select a 
new plurality of categorized information based 
on said correlation. 



8. The information presentation apparatus of claim 6 
wherein said plurality of categorized information in- 
cludes an advertisement; said correlation mecha- 
nism further comprises an advertisement correla- 
tion mechanism configured to determine an adver- 
tising correlation; and said selection mechanism 
further comprises an advertisement selection 
mechanism configured to select a new advertise- 
ment based on said advertising correlation. 

9. The information presentation apparatus of claim 6 
further comprising: 

a reading speed measuring mechanism config- 
ured to measure said user's reading speed for 
at least one of said plurality of categorized in- 
formation; and 

a time measuring mechanism configured to 
measure a time period when said gaze position 
is within a display area containing one of said 
plurality of categorized information; said time 
period and said reading speed comprising cor- 
relation parameters. 

10. The information presentation apparatus of claim 9 



12. The information presentation system of claim 11 fur- 
ther comprising: 

35 

a history mechanism configured to maintain a 
correlation history associated with said user; 
and 

a refinement mechanism configured to deter- 
mine a refined correlation using said correlation 
history. 

13. The information presentation system of claim 11 
45 wherein said plurality of categorized information in- 
cludes an advertisement; said correlation mecha- 
nism further comprises an advertisement correla- 
tion mechanism configured to determine an adver- 
tising correlation; and said selection mechanism 

so further comprises an advertisement selection 
mechanism configured to select a new advertise- 
ment based on said advertising correlation. 

14. The Information presentation system of claim 11 fur- 
55 ther comprising: 

a reading speed measuring mechanism config- 
ured to measure said user's reading speed for 



13 



BNSDOCID: <EP .0816084A2_L> 



25 



EP 0 816 984 A2 



26 



at least one of said plurality of categorized in- 
formation; and 

a time measuring mechanism configured to 
measure a time period when said gaze position 
is within a display area containing one of said 
plurality of categorized information; said time 
period and said reading speed comprising cor- 
relation parameters. 

15. The information presentation system of claim 14 
wherein said correlation mechanism additionally 
uses one or more of said correlation parameters to 
determine said correlation. 

16. A computer program product comprising: 

(a) a computer usable storage medium having 
computer readable code embodied therein for 
causing a computer to present information to a 
user; said computer readable code comprising: 

(b) computer readable code devices configured 
to cause said computer to effect a display 
mechanism configured to display a plurality of 
categorized information on a display device; 

computer readable code devices config- 
ured to cause said computer to effect a 
monitoring mechanism configured to mon- 
itor a gaze position to determine a level of 
interest of said user in at least one of said 
plurality of categorized information dis- 
played on said display device; 

computer readable code devices config- 
ured to cause said computer to effect a re- 
trieval mechanism configured to retrieve 

one or more topics classifying said catego- 
rized information displayed on said display 
device; 

computer readable code devices config- 
ured to cause said computer to effect a cor- 
relation mechanism configured to deter- 
mine a correlation between said topics and 
said level of interest of said user; and 

computer readable code devices config- 
ured to cause said computer to effect a se- 
lection mechanism configured to select a 
new plurality of categorized information 
based on said correlation. 

17. The computer program product of claim 16 further 
comprising: 

computer readable code devices configured to 



cause said computer to effect a history mech- 
anism configured to maintain a correlation his- 
tory associated with said user; and 

s computer readable code devices configured to 

cause said computer to effect a refinement 
mechanism configured to determine a refined 
correlation using said correlation history. 

10 1 8. The computer program product of claim 1 6 wherein 
said plurality of categorized information includes an 
advertisement; 

said correlation mechanism further comprises 
^5 computer readable code devices configured to 

cause said computer to effect an advertisement 
correlation mechanism configured to determine 
an advertising correlation; and 

20 said selection mechanism further comprises 

computer readable code devices configured to 
cause said computer to effect an advertisement 
selection mechanism configured to select a 
new advertisement based on said advertising 
25 correlation. 

19. The computer program product of claim 16 further 
comprising: 

30 computer readable code devices configured to 

cause said computer to effect a reading speed 
measuring mechanism configured to measure 
said user's reading speed for at least one of 
said plurality of categorized information; and 

35 

computer readable code devices configured to 
cause said computer to effect a time measuring 
mechanism configured to measure a time peri- 
od when said gaze position is within a display 
40 area containing one of said plurality of catego- 

rized information; said time period and said 
reading speed comprising correlation parame- 
ters. 

45 20. The computer program product of claim 1 9 wherein 
said correlation mechanism additionally uses one 
or more of said correlation parameters to determine 
said correlation. 



so 
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